Real - time Stereo Disparity

The dense stereo disparity map, i.e. image based distance measurement has been developed for applications such as robotic vision and video surveillance. Two small video cameras embedded in the hand-held computer or game controller can be turned into a distance image sensor or a range finder. Electrical retina stimulation with the implanted 2D electrode array that has recently been reported is potentially capable for blind people to regain vision with the technology of BMI (Brain Machine Interface). The dense disparity map is definitely one important mode of artificial vision to sense the distance by vision, when such BMI is fully developed. The challenge is to perform frame-by-frame image processing fast enough to keep up with a video rate. (1) The objective of study is to develop a robust and fast stereo matching algorithm usable in the real time video environment. To calculate and render the stereo disparity map in real time at a video rate is a challenging problem. In the approach to use two cameras posed to have their optical axes in parallel, typically local cross-correlations are measured from a point in the left image to that in the right image along the same raster scan line, then stereo correspondence is searched for every pixel by finding the maximum among the local cross-correlations. The time warp algorithm based on the dynamic programming (DP) optimizes the search for the entire raster scan line. To ensure the accuracy up to a pixel distance or even to a fractional distance, the pixel-to-pixel similarity matrix of size N2 needs to be calculated for a raster length N. This imposes a large computational burden to calculate a dense stereo disparity map, and to keep up with a video rate such as 30 frames/sec. The Dynamic Time Warp Algorithm (DTW), an implementation of Dynamic Programming (DP) is a well accepted method for the problem of stereo matching, because DTW has a time constraint that the sequence of data is retained when matching two sequences. In another word, a pixel in a sequence ismatched to a pixel after the pixel following a previouslymatched pixel in the other sequence to be matched. Therefore, DTW is particularly useful to match two raster profiles which are basically a time sequence. Since the DTW matches all pixels in the raster according to the cost function to minimize, the stereo correspondence for all pixels in the raster is calculated as a result. However, the search for matching is exhaustively done for each and every pixel in the raster, thus the process is time consuming particularly to calculate Real-time Stereo Disparity Map for Continuous Distance Sensing Applications A Method of Sparse Correspondence 25